Top 100 LLM Observability Repositories
Ranking
| Ranking | Project Name | Stars | Forks | Language | Open Issues | Description | Last Commit |
|---|---|---|---|---|---|---|---|
| 1 | langfuse | 30,834 | 3,241 | TypeScript | 302 | 🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. 🍊YC W23 | 2026-07-09 |
| 2 | mlflow | 26,957 | 5,973 | Python | 1451 | The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controllin... | 2026-07-10 |
| 3 | Self-Hosting-Guide | 21,899 | 1,098 | Dockerfile | 21 | Self-Hosting Guide. Learn all about locally hosting (on premises & private web servers) and managing software applications by yourself or your organization. Including Cloud, LLMs, WireGuard, Automati... | 2025-06-27 |
| 4 | agents-towards-production | 20,949 | 2,785 | Jupyter Notebook | 6 | End-to-end, code-first tutorials for building production-grade GenAI agents. From prototype to enterprise deployment. | 2026-07-04 |
| 5 | openobserve | 19,791 | 906 | TypeScript | 540 | Open source observability platform for logs, metrics, traces, frontend monitoring, pipelines and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and El... | 2026-07-10 |
| 6 | kubesphere | 16,994 | 2,745 | Go | 341 | The container platform tailored for Kubernetes multi-cloud, datacenter, and edge management ⎈ 🖥 ☁️ | 2026-06-25 |
| 7 | RagaAI-Catalyst | 16,143 | 3,578 | Python | 17 | Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced anal... | 2026-02-11 |
| 8 | tensorzero | 11,688 | 948 | Rust | 0 | TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation. | 2026-06-11 |
| 9 | bisheng | 11,507 | 1,879 | TypeScript | 107 | BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SF... | 2026-07-10 |
| 10 | voltagent | 10,004 | 1,039 | TypeScript | 30 | AI Agent Engineering Platform built on an Open Source TypeScript AI Agent Framework | 2026-07-10 |
| 11 | evidently | 7,678 | 875 | Jupyter Notebook | 234 | Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics. | 2026-05-02 |
| 12 | hertzbeat | 7,308 | 1,304 | Java | 306 | An AI-powered next-generation open source real-time observability system. | 2026-07-08 |
| 13 | openllmetry | 7,284 | 1,018 | Python | 107 | Open-source observability for your GenAI or LLM application, based on OpenTelemetry | 2026-07-08 |
| 14 | plano | 6,687 | 444 | Rust | 104 | Plano is an AI-native proxy and data plane for agentic apps — with built-in orchestration, safety, observability, and smart LLM routing so you stay focused on your agents core logic. | 2026-07-09 |
| 15 | helicone | 5,929 | 625 | TypeScript | 38 | 🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓 | 2026-07-05 |
| 16 | logfire | 4,357 | 256 | Python | 181 | AI observability platform for production LLM and agent systems. | 2026-07-10 |
| 17 | agenta | 4,279 | 565 | TypeScript | 118 | The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place. | 2026-07-10 |
| 18 | deepflow | 4,172 | 471 | Go | 160 | eBPF Observability - Distributed Tracing and Profiling | 2026-07-10 |
| 19 | claude-devtools | 3,685 | 280 | TypeScript | 23 | The missing DevTools for Claude Code — inspect session logs, tool calls, token usage, subagents, and context window in a visual UI. Free, open source. | 2026-05-13 |
| 20 | langwatch | 3,359 | 328 | TypeScript | 524 | The platform for LLM evaluations and AI agent testing | 2026-07-10 |
| 21 | pezzo | 3,253 | 276 | TypeScript | 42 | 🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more. | 2026-03-31 |
| 22 | holmesgpt | 2,816 | 403 | Python | 100 | SRE Agent - CNCF Sandbox Project | 2026-07-09 |
| 23 | openlit | 2,585 | 321 | TypeScript | 20 | Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, ... | 2026-07-09 |
| 24 | agentfield | 2,329 | 369 | Go | 74 | Build, run and scale AI agents like API and microservices - observable,auditable and identity-aware from day one. | 2026-07-09 |
| 25 | tokencost | 1,991 | 107 | Python | 18 | Easy token price estimates for 400+ LLMs. TokenOps. | 2025-09-05 |
| 26 | trench | 1,645 | 63 | TypeScript | 3 | Trench — Open-Source Analytics Infrastructure. A single production-ready Docker image built on ClickHouse, Kafka, and Node.js for tracking events. Easily build product analytics dashboards, LLM RAGs, ... | 2026-04-06 |
| 27 | trpc-agent-go | 1,531 | 225 | Go | 31 | A Go framework for building production agent systems with graph workflows, tools, memory, A2A, AG-UI, MCP, evaluation, and observability. | 2026-07-10 |
| 28 | llmgateway | 1,388 | 155 | TypeScript | 25 | Route, manage, and analyze your LLM requests across multiple providers with a unified API interface. | 2026-07-09 |
| 29 | future-agi | 1,357 | 353 | Python | 125 | Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0. | 2026-07-09 |
| 30 | monoscope | 1,317 | 55 | Haskell | 5 | Monoscope lets you ingest and explore your logs, traces and metrics. We store these in S3 compatible buckets. Query in natural language via LLMs. | 2026-07-09 |
| 31 | prompty | 1,227 | 118 | TypeScript | 8 | Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandab... | 2026-07-01 |
| 32 | langtrace | 1,214 | 124 | TypeScript | 0 | Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorD... | 2025-11-17 |
| 33 | superlog | 1,008 | 73 | TypeScript | 5 | Open-source observability tool that uses AI agents to self-heal your software | 2026-07-09 |
| 34 | langkit | 991 | 73 | Jupyter Notebook | 33 | 🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance m... | 2024-11-22 |
| 35 | GoModel | 991 | 70 | Go | 10 | AI gateway written in Go. Lightweight unified OpenAI-compatible API for OpenAI, Anthropic, Gemini, Groq, xAI & Ollama. LiteLLM alternative with observability, guardrails, streaming, costs and usage tr... | 2026-07-09 |
| 36 | TokenTracker | 959 | 93 | JavaScript | 4 | Track token usage across 25 AI coding tools — Claude Code, Codex, Cursor, Gemini, Kiro, OpenCode, Antigravity, Copilot, Kimi, CodeBuddy, WorkBuddy, Grok, Kilo, Roo, Zed, Goose, Mimo, ZCode & more — lo... | 2026-07-10 |
| 37 | cv-santiago | 734 | 259 | HTML | 3 | Interactive CV with AI chat integration. Built with React 19, TypeScript, Claude API. Chat with my AI avatar about my experience. | 2026-07-02 |
| 38 | aperture | 730 | 36 | Go | 50 | Rate limiting, caching, and request prioritization for modern workloads | 2025-12-21 |
| 39 | ai-gateway | 612 | 58 | Rust | 6 | The fastest, lightest, and easiest-to-integrate AI gateway on the market. Fully open-sourced. | 2025-11-21 |
| 40 | ai-shortVideo-pipeline | 572 | 22 | Python | 4 | End-to-end AI short-video production pipeline. FastAPI orchestration + Spring Boot gateway with multi-model failover, circuit breaker, metering, and full-stack observability. AI quality gating: prompt... | 2026-06-15 |
| 41 | agentsight | 518 | 71 | C | 18 | System-level AI agent profling/tracing/monitoring tool and skills in eBPF | 2026-07-09 |
| 42 | langfuse-python | 436 | 308 | Python | 5 | 🪢 Langfuse Python SDK - Instrument your LLM app with decorators or low-level SDK and get detailed tracing/observability. Works with any LLM or framework | 2026-07-09 |
| 43 | openllmetry-js | 404 | 70 | TypeScript | 25 | Sister project to OpenLLMetry, but in Typescript. Open-source observability for your LLM application, based on OpenTelemetry | 2026-06-18 |
| 44 | apo | 377 | 54 | Go | 2 | APO is a comprehensive observability platform combining OpenTelemetry with eBPF. Leveraging LLM to enable automated analysis and troubleshooting 🚀. | 2026-03-25 |
| 45 | nudgebee | 376 | 263 | Go | 49 | Unified CloudOps platform with AI-SRE, AI-FinOps, AI-K8sOps, and the Agentic Automation Builder without fragmented tools, context switching, or model lock-in. | 2026-07-09 |
| 46 | agent-prism | 370 | 21 | TypeScript | 8 | React components for visualizing traces from AI agents | 2026-07-08 |
| 47 | aigne-framework | 367 | 182 | TypeScript | 11 | The functional, composable, and typescript-first AI Agent framework for real-world LLM Apps. | 2026-01-25 |
| 48 | error-monitoring-agent | 365 | 52 | Python | 0 | Intelligent error monitoring agent that uses Airweave to find context from your code, tickets, and Slack. Clusters similar errors, enriches with relevant context, and creates actionable alerts. | 2026-01-29 |
| 49 | llmio | 310 | 37 | TypeScript | 14 | Unified LLM gateway with weighted load balancing, observability & cost tracking. 统一的 LLM 网关，提供权重负载均衡、可观测性与费用追踪。 | 2026-05-29 |
| 50 | llm-bridge | 296 | 30 | TypeScript | 1 | Interoperability between input formats of various LLMs, with observability, error handling, etc. built in. | 2026-05-31 |
| 51 | tokentelemetry | 221 | 29 | TypeScript | 3 | Token telemetry dashboard for AI autonomous and coding agents — tracks tokens, sessions, tool calls & reasoning across Hermes agent, Claude Code, Antigravity CLI, Codex & more. 100% local. | 2026-07-09 |
| 52 | langfuse-docs | 220 | 256 | MDX | 24 | 🪢 Langfuse documentation -- Langfuse is the open source LLM Engineering Platform. Observability, evals, prompt management, playground and metrics to debug and improve LLM apps | 2026-07-10 |
| 53 | agent-inspect | 219 | 88 | TypeScript | 10 | Local execution trees for TypeScript AI agents. agent-inspect helps you understand what happened inside an AI agent run — locally. It turns manual steps, tool calls, LLM calls, structured logs, failu... | 2026-07-10 |
| 54 | hub | 218 | 31 | Rust | 3 | High-scale LLM gateway, written in Rust. OpenTelemetry-based observability included | 2026-07-02 |
| 55 | knowledgeops-agent | 215 | 16 | Java | 4 | Enterprise-ready Spring AI platform for RAG, tool calling, async ingestion, JWT/RBAC security, and observability. | 2026-06-29 |
| 56 | aura | 203 | 21 | Rust | 86 | AURA is an agentic harness that turns an LLM model into a reliable, autonomous service capable of executing real SRE work. AURA provides the guardrails, API servers, state management, authentication, ... | 2026-07-09 |
| 57 | kitaru | 201 | 15 | Python | 23 | Record, replay, and improve AI agents in production, built on ZenML | 2026-07-09 |
| 58 | arconia | 174 | 20 | Java | 20 | Arconia is a framework that acts as an add-on for Spring Boot, designed to enhance modern enterprise applications built with Java. You can add Arconia to an existing Spring Boot application to boost d... | 2026-07-08 |
| 59 | ai-agent-automation | 168 | 95 | TypeScript | 18 | Modular AI agent workflow automation platform with schedulers, tools, and observability. https://vmdeshpande.github.io/ai-automation-platform-website/ | 2026-07-09 |
| 60 | AgenticGoKit | 163 | 35 | Go | 30 | Open-source Agentic AI framework in Go for building, orchestrating, and deploying intelligent agents. LLM-agnostic, event-driven, with multi-agent workflows, MCP tool discovery, and production-grade o... | 2026-07-02 |
| 61 | AgentGuard | 160 | 9 | JavaScript | 1 | Real-time guardrail that shows token spend & kills runaway LLM/agent loops. | 2025-07-31 |
| 62 | example-rag-app | 158 | 24 | TypeScript | 1 | Open-Source RAG app with LLM Observability (Langfuse), support for 100+ providers (LiteLLM), Dockerized, Full Type-checking, 100% Test coverage, and more... | 2026-01-15 |
| 63 | llemonstack | 148 | 27 | TypeScript | 1 | All-in-one local low-code AI agent development platform. Installs and runs n8n, Flowise, Browser-Use, Qdrant, Ollama, and more. Proxies LLM requests through LiteLLM with Langfuse for observability. | 2026-02-03 |
| 64 | langfuse-js | 147 | 96 | TypeScript | 6 | 🪢 Langfuse JS/TS SDKs - Instrument your LLM app and get detailed tracing/observability. Works with any LLM or framework | 2026-07-09 |
| 65 | zabbix-mcp-server | 146 | 26 | Python | 7 | MCP server for the complete Zabbix API - 237 tools, multi-server, OAuth 2.1 + bearer auth, PDF reports, systemd ready. Works with ChatGPT, Claude, VS Code, Codex, JetBrains and any MCP client. | 2026-07-09 |
| 66 | multi-agent-ralph-loop | 140 | 21 | Shell | 1 | Autonomous orchestration framework for Claude Code with MemPalace-inspired memory (4-layer stack, 818-token wake-up), parallel-first Agent Teams (6 teammates), Aristotle First Principles methodology, ... | 2026-06-22 |
| 67 | oss-llmops-stack | 139 | 7 | - | 1 | Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability,... | 2025-02-15 |
| 68 | steer | 134 | 5 | Python | 0 | The Active Reliability Layer for AI Agents. Catch failures, teach fixes, and automate reliability | 2026-01-19 |
| 69 | brain0 | 132 | 12 | Rust | 0 | The black box for AI-written code. Passive decision graph linking every commit to the agent prompts behind it: drift detection, DLP audit of what agents read, evidence-driven risk, MCP memory for codi... | 2026-07-02 |
| 70 | language-ai-engineering-lab | 126 | 27 | Jupyter Notebook | 0 | Language AI Engineering Lab, a place where you can deeply understand and build modern Language AI systems, from fundamentals to production. | 2026-06-24 |
| 71 | gopher-mcp | 122 | 18 | C++ | 78 | MCP C++ SDK - Model Context Protocol implementation in CPP with enterprise-grade security, observability and connectivity. | 2026-07-08 |
| 72 | rocketplaneIO | 122 | 1 | Go | 0 | Self-hosted AI SRE for Kubernetes — zero-instrumentation eBPF observability plus a copilot that fixes issues through guardrailed, self-verifying actions. BYO-LLM, air-gapped capable. | 2026-07-10 |
| 73 | agentwatch | 121 | 11 | Python | 0 | A powerful AI observability framework that provides comprehensive insights into agent interactions across platforms, enabling developers to monitor, analyze, and optimize AI-driven applications with m... | 2025-05-14 |
| 74 | generative-ai-project-template | 118 | 11 | Python | 2 | Template for a new generative ai project using uv, nicegui, fastapi, llms (cloud & local with litellm and ollama, cpu/gpu) and langfuse for llm observability | 2026-07-08 |
| 75 | AgC | 112 | 9 | Kotlin | 21 | AgC is the open-core platform that powers Open Agentic Compute — a new compute substrate purpose-built for deploying, running, and orchestrating AI agents at scale. | 2026-07-09 |
| 76 | tma1 | 108 | 12 | Go | 1 | Local-first observability your agent reads back. TMA1 records every LLM call, then routes what it sees into the agent's next turn via hooks and MCP. | 2026-07-03 |
| 77 | AIephant-AI-Agent-Gateway | 106 | 15 | Rust | 0 | Alephant is an open-source AI Agent Gateway for routing, tracking, and controlling LLM usage across AI agents, members, and workflows, and for publishing agent capabilities as paid endpoints with x402... | 2026-06-12 |
| 78 | JDOxyGent4J | 104 | 20 | Java | 1 | [ACL 2026] OxyGent: Making Multi-Agent Systems Modular, Observable, and Evolvable via Oxy Abstraction https://arxiv.org/abs/2604.25602 | 2026-06-30 |
| 79 | langfuse-mcp | 100 | 24 | Python | 1 | A Model Context Protocol (MCP) server for Langfuse, enabling AI agents to query Langfuse trace data for enhanced debugging and observability | 2026-06-27 |
| 80 | Your-First-LLM-Studio | 97 | 8 | TypeScript | 13 | First LLM Studio: local-first LLM studio for Apple Silicon with MLX runtimes, Compare Lab, benchmark ops, replay, and runtime telemetry. | 2026-07-03 |
| 81 | moneat | 96 | 7 | Kotlin | 19 | 2026-07-08 | |
| 82 | awesome-mcp-servers-devops | 95 | 46 | - | 7 | A curated, DevOps-focused list of Model Context Protocol (MCP) servers—covering source control, IaC, Kubernetes, CI/CD, cloud, observability, security, and collaboration—with a bias toward maintained,... | 2026-07-03 |
| 83 | altk-evolve | 95 | 11 | Python | 54 | Self improving agents through iterations | 2026-07-08 |
| 84 | infermux | 94 | 7 | Go | 0 | Route inference across providers. | 2026-07-03 |
| 85 | tokenjam | 92 | 15 | Python | 32 | Token Efficiency For AI Agents | 2026-07-09 |
| 86 | AgentLoom | 89 | 8 | Python | 4 | Simple, flexible workflow orchestration for multi-agent AI apps, with YAML configuration, runtime safety, observability, and resume support. | 2026-07-03 |
| 87 | openclaw-telemetry | 89 | 13 | TypeScript | 0 | Telemetry for OpenClaw - Captures tool calls, LLM usage, agent lifecycle, and message events. Outputs to JSONL file and optionally to syslog for SIEM integration. | 2026-05-12 |
| 88 | ollychat | 89 | 7 | TypeScript | 1 | Create custom DevOps AI agents that understand and manage your infrastructure. | 2025-02-27 |
| 89 | traccia-py | 80 | 19 | Python | 0 | OpenTelemetry-native observability, governance, and compliance for AI agents and LLM applications | 2026-06-25 |
| 90 | llm-observability | 79 | 14 | Jupyter Notebook | 0 | Learn by example how to instrument Datadog's LLM Observability product | 2026-07-09 |
| 91 | invariant-gateway | 76 | 9 | Python | 1 | LLM proxy to observe and debug what your AI agents are doing. | 2025-11-06 |
| 92 | sample-genai-on-eks-starter-kit | 76 | 43 | JavaScript | 13 | A comprehensive toolkit for deploying production-ready Generative AI infrastructure on Amazon EKS. Includes pre-configured components for: 🚀 AI Gateway (LiteLLM) 🤖 LLM Serving (vLLM, SGLang, Ollama... | 2026-07-08 |
| 93 | rag-ops | 75 | 19 | Python | 0 | This project applies the core knowledge from the LLMOps module, including the design and implementation of the API Layer, Inference Layer, Observability Layer, Cache Layer, Guardrails Layer, Routing L... | 2025-12-27 |
| 94 | lumen | 74 | 37 | Python | 0 | Lumen — learner-owned AI education platform. Tell the AI what you want to learn: it builds you a private course in ~a minute, tutors you with course-scoped RAG + citations, and lets you share, clone &... | 2026-06-07 |
| 95 | NeMo-Relay | 73 | 39 | Rust | 3 | Multi-language agent runtime and library for execution scope management, lifecycle events, and middleware on tool and LLM calls. | 2026-07-10 |
| 96 | spark-dashboard | 73 | 14 | TypeScript | 3 | Real-time hardware and LLM inference monitoring — GPU, CPU, memory, and vLLM metrics streamed to a dashboard. | 2026-07-09 |
| 97 | otelite | 73 | 1 | Rust | 28 | Lightweight OpenTelemetry receiver and local dashboard for LLM development — single binary, zero dependencies | 2026-06-22 |
| 98 | c-sentinel | 72 | 8 | C | 6 | Semantic Observability for UNIX Systems - A lightweight C-based system prober with AI-powered analysis | 2026-03-28 |
| 99 | agentcanvas | 70 | 7 | Python | 0 | Visualize Pydantic AI agent workflows from Logfire traces as an interactive HTML diagram — tools, nested sub-agents, tokens and exact cost. | 2026-06-17 |
| 100 | RAG-Multimodal-Financial-Doc-Analysis-and-Recall | 68 | 17 | Python | 0 | Production-grade multimodal RAG for financial document intelligence. Chart understanding · hybrid retrieval · numeric guardrails · multi-tenancy · full observability. | 2026-07-09 |